Skip to content

fix(release): recover immutable failed tag - #542

Merged
jeremi merged 1 commit into
mainfrom
codex/v015-recover-published-tag
Jul 28, 2026
Merged

jeremi merged 1 commit into
mainfrom
codex/v015-recover-published-tag

Conversation

@jeremi

@jeremi jeremi commented Jul 28, 2026

Copy link
Copy Markdown
Member

Summary

  • add a default-branch-only repository_dispatch: release_recovery path to the existing no-rebuild release workflow
  • recover an already-immutable tag by checking out that exact annotated tag and keeping the normal candidate, receipt, scan, provenance, and prewrite checks
  • normalize the single extra newline Git exposes for annotated tag contents
  • exclude non-promoted platform CLI report artifacts before the tag's older closed inventory validator runs
  • keep normal tag-push promotion strict: its tag target must still equal current origin/main

Why

The immutable v0.15.0 tag was pushed with a valid candidate binding to successful sealed candidate run 30376967009. Publication run 30381585873 failed before any writes because git for-each-ref exposed two trailing newlines while the tag-binding parser accepted at most one. The tag must not be moved or recreated.

This recovery uses workflow code only from protected default branch, checks out the exact existing tag, performs no rebuild, and runs the same publication jobs. workflow_dispatch remains forbidden.

Verification

  • existing v0.15.0 annotated tag parses with the corrected parser
  • python3 release/scripts/test_release_candidate.py (20 passed)
  • python3 release/scripts/test_registry_release.py (92 passed)
  • python3 release/scripts/check-gates-inventory.py (168 gates)
  • workflow YAML parses with PyYAML
  • git diff --check

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
@jeremi
jeremi marked this pull request as ready for review July 28, 2026 17:27
@jeremi
jeremi enabled auto-merge (squash) July 28, 2026 17:27
@jeremi
jeremi merged commit cef8fb9 into main Jul 28, 2026
29 checks passed
@jeremi
jeremi deleted the codex/v015-recover-published-tag branch July 28, 2026 17:35

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 402056ea74

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +116 to +119
message = Path(sys.argv[1]).read_text(encoding="utf-8")
if message.endswith("\n\n"):
message = message[:-1]
binding = release_candidate.parse_tag_binding(message)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Normalize the message passed to every tag verifier

In the repository_dispatch recovery path, the later verify-candidate job checks out the immutable tag target, so it executes that tag's old release_candidate.py rather than the parser changed in this commit. This normalization fixes only the initial inline parse; lines 373–376 recreate the raw two-newline tag message and pass it to the old \n? parser, reproducing the exact failure before publication. The normalized message must be carried into downstream verification, or the fixed verifier must come from trusted default-branch code.

Useful? React with 👍 / 👎.

Comment on lines +7 to +8
repository_dispatch:
types: [release_recovery]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Align dispatch-generated evidence with its branch identity

A repository_dispatch run receives the default-branch ref, and checking out the requested tag does not change the OIDC identity used by cosign or the source ref recorded by the SLSA generator. The workflow nevertheless verifies release.yml@refs/tags/${tag} and --source-tag "${tag}" at lines 1152–1171, matching the public contract in release/VERIFY.md; after the images and GitHub Release have been written, reconciliation will reject the branch-identity certificates/provenance and leave a failed, partially completed promotion.

Useful? React with 👍 / 👎.

Comment on lines +143 to +146
if [[ "${GITHUB_EVENT_NAME}" != "repository_dispatch" ]]; then
test "$(git rev-parse refs/remotes/origin/main)" = \
"${{ steps.release.outputs.tag_target }}"
fi

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Require proof of failure before relaxing main equality

For every repository_dispatch, this skips the exact-current-main check and retains only the ancestor check. An operator can therefore push a new annotated tag at any still-valid older candidate commit and dispatch recovery: the normal tag-push run rejects that stale target, but this path accepts and publishes it. Because the payload is not constrained to the known failed tag and no prior failed prewrite run is verified, the exception is broader than recovery of an already-immutable failed tag.

Useful? React with 👍 / 👎.


concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
group: ${{ github.workflow }}-${{ github.event.client_payload.tag || github.ref }}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Use one concurrency key for both release entry points

For the same tag, a tag-push run uses a key ending in refs/tags/vX.Y.Z, while a recovery dispatch uses the bare payload vX.Y.Z, so GitHub does not serialize them despite cancel-in-progress: false. If recovery is dispatched while the original tag-push run or its rerun is still active, both workflows can pass the prewrite absence checks and enter the package and release write stages concurrently; normalize both event paths to the same tag-name key.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant